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MEMORANDUM  FOR  UNDER  SECRETARY  OF  DEFENSE  (ACQUISITION  & 
TECHNOLOGY) 

SUBJECT:  Report  of  the  Defense  Science  Board  (DSB)  Task  Force  on 

Concurrency  and  Risk  of  the  F-22  Program 


I  am  pleased  to  forward  the  final  report  of  the  DSB  study  on 
the  F-22  program.  This  study  was  conducted  at  your  request  to 
assess  the  degree  of  concurrency  and  risk  in  the  F-22  program  as 
directed  by  the  Senate  Armed  Services  Committee  in  the  SASC 
Report  163-282,  dated  14  June  1994. 

The  F-22  is  the  next  generation  air  superiority  fighter  with 
many  "first  time"  features  (super  cruise,  vectored  thrust, 
stealth,  passive  location  systems,  integrated  avionics)  making  it 
one  of  the  most  challenging  developments  undertaken.  The  overall 
program  has  been  structured  to  address  all  these  areas  in  an 
orderly  and  appropriate  way. 


The  Task  Force  concluded  that  the  critical  point  in  the 
program  is  the  ramp-up  in  the  production  from  4  aircraft  to  12 
aircraft  per  year.  This  is  also  the  time  the  engine  and  the 
passive  surveillance  avionics  subsystem  have  the  greatest  number 
of  uncertainties.  The  Task  Force  identified  for  each  risk  area 
specific  significant  events  whose  accomplishment  should  take 
place  before  contract  award  for  the  12  aircraft  buy. 


I  concur  with  the  recommendations  of  the  Task  Force  and 
recommend  that  you  forward  the  Report  to  the  Secretary  of 


Defense . 


Craig  I . 
Chairman 
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OFFICE  OF  THE  SECRETARY  OF  DEFENSE 

WASHINGTON,  D  C.  20301-3140 


DEFENSE  SCIENCE 
BOARD 


MEMORANDUM  FOR  CHAIRMAN,  DEFENSE  SCIENCE  BOARD 


SUBJECT:  Final  Report  of  the  Defense  Science  Board  Task  Force  on 

Concurrency  and  Risk  of  the  F-22  Program 


I  am  pleased  to  forward  the  final  report  of  the  Task  Force 
on  Concurrency  and  Risk  of  the  F-22  Program. 

The  F-22  is  the  next  generation  air  superiority  fighter 
scheduled  for  introduction  in  the  first  decade  of  the  next 
century.  The  design  incorporates  major  advances  in  all  areas  of 
performance  including:  stealth,  supercruise,  vectored  thrust, 
situation  awareness  and  integrated  avionics.  It  is,  thus  a  very 
ambitious,  challenging  program,  probably  the  most  technically 
challenging  program  in  recent  times . 

In  response  to  USD(A&T)  tasking,  the  Task  Force  examined  the 
concurrency/risk  in  the _ F-22  program  plan.  We  found  the  existing 
progr^  plan  to  be  detailed,  logical  and  orderly.  We  reviewed 
the  risks  in  each  major  area  and  identified  for  each  area  those 
key  events  whose  accomplishment  was  needed  to  provide  assurance 
that  the  program  was  ready  to  move  into  significant  production 
rate,  from  4  to  12  a/c  per  year.  Further,  each  of  these  "events" 
has  a  series  of  preceding  accomplishments  which  given  adequate 
forewarning  of  potential  problems. 

We  also,  as  requested,  reviewed  several  studies  that  compare 
the  F-22  schedule  with  that  of  previous  fighter  programs  and 
concluded  that  the  F-22  plan  is  consistent  with  and,  in  many 
ways,  is  more  conservative  than  these  other  programs. 

Our  conclusion,  thus,  is  that  the  F-22  plan  has  acceptable 
overlap  of  development  and  production  (concurrency)  and  that  the 
risks  associated  with  premature  entry  into  rate  production  are 
readily  controllable  through  insistence  on  meeting  the  key 
"events"  identified  in  this  report.  Thus,  there  is  no  reason 
based  upon  risk/concurrency  considerations  to  introduce  a  program 
stretch  at  this  time.  If  other  reasons  cause  some  program 
stretch,  special  attention  and  support  should  be  given  to  the 
engine  and  the  passive  location  subsystem  (a  key  element  of  the 
situation  awareness  system) ,  the  areas  with  the  greatest  number 
of  uncertainties  at  this  time. 

The  Task  Force  would  like  to  stress  one  point:  there  are 
substantial  margins  throughout  the  F-22  specifications  and  a  very 
capable  aircraft  would  result  even  if  performance  fell  somewhat 
short  of  meeting  many  or  even  all  of  these  specs.  It  is  therefore 


important  that  the  SPO  and  OSD  hot  take  a  rigid  stance  on  meeting 
all  specs  but  rather,  as  the  program  progresses,  look  at  the 
overall  performance,  cost  and  schedule  impacts  in  deciding  which 
if  any,  perfoinnance  areas  need  further  work. 

The  Task  Force,  given  the  very  short  time  available,  limited 
its  considerations  to  the  "concurrency/risk"  issue  called  for  in 
the  Terms  of  Reference.  I  would,  however,  like  to  offer  some 
personal  views  regarding  this  very  important  and  large  program. 

1.  The  world's  air  forces  are  starting  to  and  will 
eventually  catch  up  with  the  F-15/16/18  capabilities.  Thus, 
the  US  will  need  to  make  a  "quantum  step"  improvement  in  the 
capability  assuming  we  wish  to  retain  the  "overwhelming 
technological  superiority"  that  served  us  so  well  in  the 
Gulf  War.  We  can  start  on  this  step  now,  as  the  F-22 
program  is  doing,  or  later.  The  worst  course  of  action  is 
to  invest  heavily,  then  do  a  major  stretch  or  cancel. 

2.  The  US  has  invested  about  $60B  on  stealth  aircraft 
which  has  given  us  a  unique  and  incredibly  important 
military  advantage.  Our  total  assets,  however,  consist  of 
50  F-117s  and  20  B-2s.  The  F-22  offers  the  opportunity  to 
expand  this  advantage  substantially. 

3 .  Because  of  the  great  complexity  and  related  cost  of 
the  system  and  given  its  development  status,  it  would  be 
timely  to  do  a  careful  "scrub"  of  the  subsystems  to 
determine  what  savings  could  be  accomplished  by  simplifying, 
postponing  or  deleting  some  items  or  features  (eg.,  the 
intra-flight  communications  system. ) 

4 .  Although  some  steps  in  the  right  direction  have 
been  taken,  the  F-22  contract  and  management  approaches  do 
not  begin  to  reflect  the  acquisition  "streamlining"  (and 
resultant  cost  savings)  advanced  by  the  Secretary  of 
Defense.  Given  that  this  program  will  account  for  a 
substantial  part  of  the  Air  Force  budget  for  many  years,  a 
major  effort  in  this  area  is  strongly  suggested. 

The  Task  Force  would  like  to  thank  the  OSD  and  IDA  staff  who 
ably  supported  us  and  to  acknowledge  their  hard  work  and 
contributions.  Additionally,  we  want  to  thank  the  Air  Force  and 
the  contractors  for  their  patience  and  complete  cooperation.  We 
could  not  have  done  the  task  without  the  help  of  all  these 
people . 


Charles  A.  Fowler 
Chairman 
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EXECUTIVE  SUMMARY 


INTRODUCTION 


PURPOSE 

The  Defense  Science  Board  Task  Force  on  Concurrency  and  Risk  of  the  F-22  Program^  was 
convened  to  respond  to  concerns  raised  in  a  Senate  Armed  Services  Report  (SASC  Report  163-282,  dated 
14  June  1994.)  The  Committee  requested  that  the  Department  of  Defense  address  their  concerns  related  to 
F-22  Program  schedule  concurrency.  The  terms  of  reference  (TOR)  for  the  Task  Force  from  USD(A&T) 
contain  three  questions.^  The  questions  are: 

•  Are  there  are  any  areas  in  the  F-22  program  of  excessive  concurrency?  What  is  the  risk 
in  each  area? 

•  For  any  areas  of  identifiable  high  risk,  are  viable  plans/options  available  that  would 
mitigate  the  risk? 

•  What  conclusions  regarding  F-22  concurrency  and  risk  can  be  drawn  by  comparisons  to 
existing  data  (e.g.,  an  ongoing  IDA  study)  on  previous  fighter/combat  aircraft? 

TASK  FORCE  APPROACH 

The  Task  Force  first  met  on  3  November  1994,  briefed  its  interim  findings  to  DUSD(S&TP)  on 
7  December  and  to  USD(A&T)  on  20  December,  and  published  its  interim  report  on  3 1  December.  During 
the  November-December  1994  time  period  the  Task  Force  met  as  a  group  six  times. ^  Briefings  and 
information  were  received  from  the  Program  Office  and  other  sources  relevant  to  answering  the  questions. 
A  visit  was  made  to  the  prime  contractor,  Lockheed  Aeronautical  Systems  Company  in  Marietta,  GA,  for 
further  briefings  and  discussions,  and  to  observe  hardware  and  software  demonstrations.  Also  present  at 
that  time  were  the  engine  contractor,  Pratt  &  Whitney,  and  selected  major  subcontractors.  The  members, 
also  working  as  subgroups,  made  use  of  other  information  and  available  reports. 

The  Task  Force  re-ordered  the  TOR  questions  in  structuring  this  report,  with  the  last  question 
above  on  comparing  concurrency  in  the  F-22  with  other  fighter  aircraft  programs  addressed  first,  the 
question  on  areas  of  concurrency  and  risk  addressed  second,  and  the  question  on  mitigating  risk  addressed 
last. 

F-22  PROGRAM  DESCRIPTION 

The  F-22  is  the  next  generation  air  superiority  fighter  scheduled  for  introduction  in  the  first 
decade  of  the  21st  century.  Its  primary  role  is  to  counter  emerging  proliferating  worldwide  threats,  and  to 
maintain  the  overwhelming  air  superiority  advantage  needed  by  a  downsized  and  reshaped  U.S.  military 
force.  The  F-22  is  designed  to  penetrate  enemy  airspace  and  achieve  a  first-look,  first-kill  capability 
against  multiple  aircraft  targets.  The  features  that  will  permit  the  F-22  to  achieve  this  advantage  are:  a  low 
observable,  highly  maneuverable  airframe  making  extensive  use  of  composites  and  low  observable 
materials;  advanced  software-intensive  integrated  avionics;  improved  situational  awareness;  and  a  new 
engine  with  a  two-dimensional  vectoring  nozzle  that  is  also  capable  of  supersonic  cruise  without  using 
afterburner. 

The  advanced  nature  of  the  F-22  should  be  considered  in  the  context  of  the  prior  risk  reduction 
effort  achieved  during  an  extensive  Demonstration  and  Validation  (DEMWAL)  program,  which  included 
competitive  flying  prototypes  of  the  airframe/engine  configurations,  avionics  flying  testbeds,  and  brass 
board  components. 

The  F-22  has  been  in  Engineering  and  Manufacturing  Development  (EMD)  since  August  1991. 
First  flight  is  now  estimated  to  be  February  1997.  The  schedule  used  for  the  examination  of  the  F-22 
Program  is  the  latest  provided  to  the  Task  Force  (Premise  Modification  to  Master  Schedule  No.  17,  dated 
16  November  1994). 


^  The  Task  Force  members  are  listed  in  Appendix  A. 

2  The  Terms  of  Reference  are  presented  in  Appendix  B. 
^  The  Task  Force  schedule  is  presented  in  Appendix  C. 
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SUMMARY  FINDINGS  AND  CONCLUSIONS 


Findings— Question  1:  What  conclusions  regarding  F-22  concurrency  and  risk  can  be  drawn  by 
comparisons  to  existing  data  (e.g.,  an  ongoing  IDA  study)  on  previous 
nghter/combat  aircraft  programs? 

answer  this  question  it  was  necessary  first  to  define  concurrency  and  then  to  compare  suitable 
F-22  concurrency  measures  to  other  fighter  aircraft  programs.  In  its  report  to  Congress  on  concurrency  in 
Apnl  1990,  DoD  defined  concurrency  and  the  definition  in  that  report  is  the  definition  the  Task  Force  used. 

Befipition  of  Copcwrrepcy;^  Concurrency  is  the  degree  of  overlap  between  the 
development  and  production  processes  of  an  acquisition  program.  Structuring  the  overlap  is  a 
management  judgment  balancing  risk  of  proceeding,  cost  of  delay,  work  force  inefficiency,  design 
obsolescence,  and  operational  impact. 

It  is  important  to  note  that  concurrency  is  the  degree  of  overlap,  and  degree  must  be  measured  in 
terms  of  trade-offs  and  judgments  of  technical,  schedule,  and  cost  risk. 

'^e  F-22  was  compared  to  other  relevant  fighter  aircraft  programs  through  examination  of  studies 

performed  by  the  Institute  for  Defense  Analyses  and  RAND  Corporation,  and  through  review  of  other 
sources  of  data.^ 

Several  F-22  Program  Development  Test  and  Evaluation  (DT&E)  and  Initial  Operational  Test  and 
Evaluation  (lOT&E)  schedule  duration,  flight  hour  activity,  and  production  Lot  quantity  metrics  were 
compared  with  the  F-15,  F-16,  and  F/A-18  fighter  aircraft  programs.  The  Task  Force  found  that  the  degree 
of  concurrency  for  the  F-22  Program  as  measured  by  the  data  now  available  is  conservative  when 
compared  to  the  other  tactical  fighters.  The  key  dates  are  long-lead  and  contract  award  for  Lot  2  (12 
aircraft)  by  which  point  key  accomplishments  will  have  been  demonstrated  in  all  major  areas  The  Task 
Force  did  not  think  that  completion  of  dedicated  lOT&E  was  necessary  before  proceeding  to  production 


Findings  Question  2.  Are  there  any  areas  in  the  F-22  program  of  excessive  concurrency?  What  is 
the  risk  in  each  area? 

To  assess  the  concurrency  of  the  F-22  program  in  isolation  does  not  give  a  complete  picture  of  the 
potential  technical  or  programmatic  risks  in  the  program.  Therefore,  the  Task  Force  conducted  a  review  of 
the  technical  and  management  status  of  the  program  in  several  key  areas.  This  was  done  in  an  effort  to 
relate  concurrency  and  technical  risk. 

Lot  2  (12  aircraft)  Long-Lead  and  especially  Lot  2  Contract  Award  were  considered  critical 
milestones  They  were  chosen  as  the  critical  milestones  to  balance  schedule  and  cost  commitment  while 
ensurmg  the  fieldmg  of  the  system  in  a  reasonable  time.  The  two  dates  span  calendar  year  1 999 

Key  decision  points  were  then  identified  for  each  of  the  following  critical-technical  areas  and 
related  to  the  Lot  2  milestones: 

•  Airframe — Static  and  Fatigue  Tests  Completion;  Flight  Test  (Aircraft  4001) 

•  Engine— Redesigned  Engine  Ground  Test  and  First  Flight  Engine  Deliveiy  (Aircraft  400 1 ) 

•  Low  Observables— Pole  Model  and  Flight  Testing  (Aircraft  4004) 

•  Radar — Hardware  and  Software  Flight  Testing  (Aircraft  4005) 

•  Passive  Surveillance  and  Electronic  Warfare— Hardware  and  Software  Flight  Testing 
(Aircraft  4006) 

•  Sensor  Control/Sensor  Fusion/Pilot  Interface — ^Hardware  and  Software  Flight  Testing 
(Aircraft  4008) 

•  Flight  ControWehicle  Management— Early  Flight  Testing  (Aircraft  4001) 

•  Software— Block  0- 1  -2-3  Completion  Dates 

Acquisition  key  events  for  the  F-22  program  are  presented  in  Figure  ES-1 . 


See  Appendix  D  for  citation. 
See  Appendix  D  for  citations. 
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Findings — Question  3:  For  any  areas  of  identifiable  high  risk,  are  viable  plans/options  available 

that  would  mitigate  the  risk? 

The  Task  Force  addressed  the  question  of  viable  plans/options  available  to  mitigate  high  risk 

areas. 

We  found  that  there  were  no  alternative  completely  independent  approaches  being  pursued  for  the 
major  subsystems  (such  as  the  engine  or  the  radar),  and  we  concluded  that  such  alternative  approaches 
were  neither  practical  nor  needed. 

There  are,  of  course,  alternative  approaches  being  pursued  or  reviewed  at  the  part/component 
level  (e.g.,  turbine  blades);  and  others  (e.g.,  LO  CNl  antennas)  can  be  accommodated  within  the  program’s 
plan  and  schedule. 

We  concluded  that  the  best  approach  to  handling  problems  that  do  arise  is  to  seriously  consider 
accepting  some  reduced  F-22  performance,  slip  in  the  schedule,  or  some  combination  of  both.  As  has  been 
noted  elsewhere  in  this  report,  F-22  overall  performance,  even  if  performance  fell  short  of  that  specified  in 
many  subareas,  would  still  represent  a  major  increase  in  military  capability.  In  the  event  of  significant 
technical  problems,  rigid  adherence  to  the  current  specifications  and  goals  would  generally  be 
unproductive  and  probably  very  costly. 
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CONCLUSIONS 

With  its  many  “first  time”  features  for  a  fighter  (super  cruise,  vectored  thrust,  stealth,  passive 
location  system,  integrated  avionics),  the  F-22  is  one  of  the  most  challenging  developments  undertaken  in 
recent  times.  The  overall  program,  starting  with  the  prototype  in  the  DemonstrationA^alidation  Phase  and 
related  R&D  efforts,  has  been  structured  to  address  all  these  areas  in  an  orderly,  appropriate  way  and  much 
has  been  accomplished. 

From  the  standpoint  of  risk/concurrency,  the  Task  Force  concluded  that  the  critical  point  in  the 
program  is  the  ramp-up  in  the  production  program  from  4  aircraft  to  12  aircraft  per  year  (Lot  2). 

The  Task  Force  identified,  for  each  major  system  and  subsystem  risk  area,  specific  key  events 
whose  accomplishment  should  take  place  before  release  of  Contract  Award  (and  to  a  lesser  degree  Long- 
Lead)  funding  for  Lot  2,  the  12  aircraft  buy.  (See  Figure  ES-1). 

At  this  time  the  engine  and  the  passive  surveillance  avionics  subsystem  have  the  greatest  number 
of  uncertainties.  Consequently,  if  there  are  changes  in  the  program  plan,  these  two  areas  should  receive 
special  attention  and  continued  support. 

If  there  are  significant  delays  in  accomplishment  of  these  key  events,  or  if  the  performance  levels 
achieved  are  unacceptable,  the  program  could  be  adjusted  by  staying  at  the  4  aircraft  per  year  production 
rate  for  an  additional  year.  It  is  noted  that,  while  stretching  a  program  may  reduce  risk,  if  stretching  is  not 
necessary  it  can  result  in  increased  total  costs,  possible  loss  of  key  manpower  and  suppliers,  and  earlier 
technical/operational  obsolescence. 

After  careful  review  of  the  program  plans  and  accomplishments,  the  Task  Force  concluded  that 
there  are  appropriate  future  milestones  on  which  to  judge  readiness  for  production  ramp-up.  Thus,  there  is 
no  reason  based  upon  risk/concurrency  to  introduce  a  schedule  stretch  at  this  time. 
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REPORT  OF  THE  DEFENSE  SCIENCE  BOARD 
TASK  FORCE  ON  F-22  PROGRAM 
CONCURRENCY  AND  RISK 


TASKING 

•  What  conclusions  regarding  F-22  concurrency  and  risk  can  be  drawn  by  comparisons 
to  previous  fighter  aircraft  programs? 

•  Are  there  any  areas  in  the  F-22  Program  of  excessive  concurrency?  What  is  the  risk  in 
each  area? 

•  For  any  areas  of  identifiable  high  risk,  are  viable  plans/options  available  that  would 
mitigate  the  risk? 

Terms  of  Reference;  USD(A&T);Nov  1,  1994 


The  Task  Force  reviewed  the  F-22  Program  and  its  schedule  in  considerable  detail  in  formulating  an 
approach  to  the  tasking. 

The  Terms  of  Reference  (see  Appendix  B)  included  three  tasks  as  noted  above.  The  first  task,  reviewing 
several  studies  comparing  the  concurrency  in  the  F-22  program  plan  with  that  of  other  fighter  aircraft 
programs,  is  covered  in  pages  5-14. 

The  second  task,  reviewing  the  program  and  determining  and  identifying  the  risk  in  each  major  area,  is 
covered  in  pages  15-33. 

The  third  task,  relating  to  plans/options  to  mitigate  risk  in  key  areas,  is  covered  on  page  34. 
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F-22  KEY  FEATURES 


Super  Cruise 
Vectored  Thrust 
Stealth 

Situation  Awareness 
integrated  Avionics 

The  F-22  is  the  next  generation  air  superiority  fighter  scheduled  for  introduction  in  the  first  decade  of  the 
21st  century.  Its  primary  role  is  to  counter  emerging  proliferating  worldwide  threats,  and  to  maintain  the 
overwhelming  air  superiority  advantage  needed  by  a  downsized  and  reshaped  U.S.  military  force.  The 
F-22  is  designed  to  penetrate  enemy  airspace  and  achieve  a  first-look,  first-kill  capability  against  multiple 
aircraft  targets.  The  features  that  will  permit  the  F-22  to  achieve  this  advantage  are:  a  low  observable 
highly  maneuverable  airframe;  extensive  use  of  composites  and  low  observable  materials;  advanced 
software-intensive  integrated  avionics  providing  improved  situational  awareness;  and  a  new  engine  with  a 
two-dimensional  vectoring  nozzle  that  is  also  capable  of  supersonic  cruise  without  using  afterburner. 
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The  DSB  scheidule  concurrency  and  risk  assessment  was  based  upon  the  above  F-22  program  schedule 
resulting  from  the  Air  Force  FY1996  POM  submission. 
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TASK  FORCE  APPROACH 

•  Define  Concurrency 

•  Compare  F-22  development  schedule  concurrency  metrics  to  experiences  of  earlier 
tactical  aircraft  developments 

•  Conduct  top-level  critical  technical  and  management  areas  review  of  F-22 

•  Identify  key  decision  points  for  each  critical  technical  area 

•  Assess  F-22  degree  of  concurrency  and  risk 

The  Task  Force  addressed  concurrency  metrics  first  and  then  conducted  the  technical  review. 
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DEFINITION  AND  COMPARISONS  OF  CONCURRENCY 

•  Definition* 

”  Concurrency  is  the  degree  of  overlap  between  the  development  and  production  processes  of  an 
acquisition  program.  Structuring  the  overlap  is  a  management  judgment  balancing: 

»  Risk  of  proceeding 

»  Cost  of  delay,  work  force  inefficiency,  design  obsolescence 

•  Comparison  of  F-22  concurrency  with  other  relevant  programs 
“  IDA  Study 

--  RAND  Study 
-  Unpublished  Source 

•  Source:  DoD  Report  on  Concurrency  in  Major  Acquisition  Programs;  April  1990 


Definition  of  Concurrency: 

A  number  of  definitions  of  concurrency  are  used  by  analysts,  program  managers,  and  program  evaluators. 
For  its  report  on  “  . . .  Concurrency  in  Major  Acquisition  Programs,”*  submitted  to  Congress  in  April  1990, 
the  Department  of  Defense  chose  to  define  concurrency  as  follows: 

''Concurrency*'  is  the  degree  of  overlap  between  the  development  and  production 
processes  of  an  acquisition  program.  Structuring  this  degree  of  overlap  is  a  management 
judgment;  a  balancing  of 

•  the  risk  of  proceeding  with  certain  production  and  other  activities  weighted  against 
the  delay 

•  known  cost  of  delay  caused  by  work  force  inefficiency 

The  DSB  Task  Force  used  this  definition  in  evaluating  concurrency  in  the  F-22  program.  The  definition 
both  implicitly  and  explicitly  states  that  concurrency  in  a  major  acquisition  program  calls  for  judgment. 
Concurrency  is  the  degree  of  overlap,  and  implicitly  degree  is  based  on  judgment.  Explicitly  the  degree  of 
overlap  is  a  management  judgment  measured  in  terms  of  trade-offs  and  risks. 

In  addition,  the  Task  Force  was  asked  to  address  whether  there  were  any  areas  of  excessive  concurrency  in 
the  F-22  program.  The  concept  of  excessive  concurrency  is  a  further  judgment,  understanding  that  some 
concurrency  is  prudent  and  that  the  risks  of  proceeding  are  often  best  evaluated  in  hindsight. 

Concurrency  Metrics: 

A  number  of  metrics  could  be  used  in  comparing  the  F-22  development  schedule  concurrency  to 
experiences  with  earlier  tactical  aircraft  developments.  The  DSB  Task  Force  drew  on  three  sets  of 
metrics — one  used  by  IDA,  one  by  RAND,  and  one  by  an  unpublished  source.  Given  the  limited  time 
available,  the  DSB  did  not  create  yet  another  metric  or  conduct  an  independent  validation  of  the  three  sets 
of  metrics.  The  existing  metrics  seem  to  be  consistent  in  showing  that  the  F-22  concurrency  is  comparable 
to  that  of  other  programs.  The  following  paragraphs  describe  the  metrics  in  more  detail.  In  all  cases,  the 
Task  Force  used  F-22  schedule  data  from  the  Lockheed  premise  modification  to  Master  Schedule,  No.  17, 
dated  16  November  1994. 


*  The  Under  Secretary  of  Defense  (Acquisition),  Report  on  Guidelines  for  Determining  the  Degree  of 
Risk  Appropriate  for  the  Development  of  Major  Defense  Acquisition  Systems,  and  Assessing  the 
Degree  of  Risk  Associated  with  Various  Degrees  of  Concurrency;  and  Concurrency  in  Major 
Acquisition  Programs,  Washington,  D.C.,  April  1990. 
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In  its  reports  Assessing  Acquisition  Schedules  for  Tactical  Aircraft^  and  Schedule  Risk  Assessments  for 
Tactical  Aircraft,^  IDA  looked  at  several  concurrency  metrics.  For  this  DSB  Task  Force,  IDA  updated  two 
measures  of  concurrency.  The  first,  shown  in  the  chart,  compares  the  proportion  of  DT&E  flight  testing 
completed  (on  the  x-axis)  to  the  number  of  aircraft  committed  to  production  in  terms  of  Long-Lead 
commitment  (on  the  y-axis).  (Data  discussed  later  show  that  the  duration  of  DT&E  flight  testing  planned 
for  the  F-22  is  significantly  longer  than  for  prior  programs.) 

When  DT&E  is  complete  in  mid-2001,  the  40  F-22  aircraft  of  Lots  1,  2,  and  3,  plus  the  Long-Lead  items 
for  another  36  aircraft  (Lot  4),  are  scheduled  to  be  on  contract.  This  concurrency  metric  shows  that  fewer 
F-22  aircraft  have  been  committed  to  production  than  was  the  case  with  the  three  most  recent  fighter 
aircraft  programs. 


2  Institute  for  Defense  Analyses,-  Bruce  R.  Harmon,  Lisa  M.  Ward,  Paul  R.  Palmer,  Assessing 
Acquisition  Schedules  for  Tactical  Aircraft,  IDA  Paper  P-2105,  February  1989. 

^  Institute  for  Defense  Analyses,  Bruce  R.  Harmon,  J.  Richard  Nelson,  Neang  Om,  Janet  M.  Sater,  Alec 
W.  Salerno,  Schedule  Risk  Assessments  for  Tactical  Aircraft  Programs,  IDA  Document  D-1214, 
December  1992. 
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CONCURRENCY  BETWEEN  FRIGHT  TEST  AND  PRODUCTION 
IDA  Metric  2:  Value  of  Production  Aircraft  Commitment  During  DT&E 
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Proportion  of  DT&E  Flight  Testing  Completed 

Source  of  Historical  Program  Data:  IDA  Paper  P-2105,  February  1989. 

F-22  Data  from  Lockheed  Premise  Modification  to  Master  Schedule,  No.  17,  dated  16  November  1994. 
Historical  Financial  Data  from  DoD  Sources. 

F-22  Financial  Data  from  F-22  SPO. 


Looking  at  a  different  metric,  IDA  compared  the  proportion  of  DT&E  flight  testing  completed  to  the  dollar 
value  of  aircraft  committed.  While  the  F-22  is  more  expensive  per  aircraft,  it  has  a  profile  similar  to  the 
dollar  commitments  on  the  other  programs.  The  last  commitment  of  funding,  the  commitment  that  raises 
the  F-22  value  higher  than  those  for  the  other  three  programs,  comes  very  late  in  the  flight  test  program. 
There  is,  as  will  be  seen  in  the  following  discussion,  a  considerable  amount  of  time  during  which  the  DoD 
can  delay  making  this  last  commitment  if  flight  testing  has  shown  difficulties.  Thus,  from  the  perspective 
of  these  two  IDA  metrics,  the  F-22  compares  favorably  with  the  other  programs  in  terms  of  the  degree  of 
overlap. 
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LENGTH  OF  KEY  PERIODS  FOR  FIGHTER/ATTACK  AIRCRAFT 


**  Also  had  prototype  in  program 

Source  of  Historical  Program  Data:  IDA  Paper  P-2105,  February  1989. 

F-22Data  from  Lockheed  Premise  Modification  to  Master  Schedule,  No.  17,  dated  16  November  1994. 

Continuing  to  use  IDA  date  for  historical  programs,  this  chart  displays  the  key  milestones  for  the  F-22  and 
our  other  fighter/atteck  aircraft.  In  particular,  the  chart  shows  the  duration  of  various  testing.  Static  and 
fatigue  durations  are  slightly  below  average.  However,  flight  testing  is  planned  over  a  longer  duration— 
twice  the  average— potentially  giving  time  to  correct  problems.  Time  from  EMD  start  to  first  production  is 
also  planned  to  be  longer  than  average,  again  giving  an  opportunity  to  correct  any  problems  encountered. 
Long-Lead  release  for  Lot  1  is  after  first  flight.  Thus  in  these  categories,  the  F-22  time  lengths  seem 

reasonable  compared  with  those  of  other  tactical  aircraft  programs,  particularly  in  light  of  the  small  number 
of  aircraft  in  Lot  1. 
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FLIGHT  TEST  DATA  COMPARISONS 


Average  Hours 

Number  Air  Vehicle  Avionics  Armaments  Total  Average  Per  A/C 

A/C  Test  Hours  Test  Hours  Test  Hours  R&M  Flight  Hours  Hours/Flight  Per  Month 


F-15A/B 

819 

372 

- 

2,856 

0.94 

10.9 

F-1 6** 

489 

174 

280 

2,581 

1.24 

16.5 

F/A-18 

12 

2,365 

591 

952 

1.43 

16.2 

AV-8B** 

5 

1,037 

i 

! 

478 

523 

- 

2,038 

1.05 

13.9 

F-22** 

9 

2,426 

2,765 

_ 

- 

5,191 

1.8 

23.3 

*  The  higher  number  of  test  aircraft  for  the  F-1 5  might  be  explained  by  the  fact  that  there  was  no  prototype. 

**  Also  had  prototype  in  program 

*  The  higher  number  of  test  aircraft  for  the  F-1 5  might  be  explained  by  the  fact  that  there  was  no  prototype. 

**  Also  had  prototype  in  program 

Source  of  Historical  Program  Data:  IDA  Paper  P-2 105,  February  1989. 

F-22  Data  from  Lockheed  Premise  Modification  to  Master  Schedule,  No.  17,  dated  16  November  1994. 

Continuing  to  use  IDA  data,  the  Task  Force  examined  flight  test  data.  The  above  chart  displays  the 
scheduled  F-22  flight  test  hours — ^which  exceed  those  of  the  other  programs.  It  should  also  be  noted  that 
the  avionics  test  hours  are  planned  to  be  from  three  to  five  times  greater  than  avionics  test  hours  for  prior 
programs. 

The  planned  average  hours  per  flight  and  per  aircraft  are  higher  than  those  for  other  programs  because  of 
the  high  number  of  flight  hours  and  the  relatively  small  number  of  test  aircraft  (nine).  The  1.8  hours  per 
flight  exceeds  the  1.6  hours  per  flight  average  in  the  YF-22  test  program  because  of  a  planned  heavy  use  of 
in-flight  refueling.  The  23.3  hours  per  month  is  calculated  on  a  basis  equivalent  to  the  historical 
programs — Lockheed  plans  approximately  25  hours  per  month.  Given  Lockheed’s  optimistic  estimate  of 
flight  hours  per  month,  a  sensitivity  analysis  is  in  order.  At  19  hours  per  month — ^20%  of  the  flight  test 
hours  would  be  accomplished  by  the  time  Lot  2  Long-Lead  items  are  procured,  compared  with  30%  in  the 
current  plan.  The  F-22  program  does  have  an  accelerated  maturation;  the  Program  Office  attributes  this  to 
Demonstration/  Validation  (DEM/VAL)  benefits  as  well  as  structural  integrity  program  benefits. 
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RAND  STUDY  ON  F-22  PRODUCTION  DECISION 
(} 989-1991;  Updated  November  1994) 

Issue: 

•  When  should  production  commitments  be  authorized? 

How  much  system  maturity  is  enough? 

Approach: 

•  Link  decisions  to  development  flight  test  results 

•  Distinguish  between  kinds  of  problems 

-  Potential  program  killers 

-  Troublesome  but  manageable 

-  Routine 

Examine  database  of  prior  programs 


Whether  the  timeline,  number  of  aircraft,  and  funding  commitments  are  consistent  with  other  fighter 
aircraft  programs,  the  question  remains:  “How  much  testing  is  enough  to  reduce  risk  to  an  acceptable  level 
before  committing  to  production?”  A  RAND  project  memorandum,"^  updated  in  November  1994, 
discussed  this  issue.  Quoting  from  the  preface  of  this  Report: 

*‘The  Congress  has,  over  the  past  decade,  issued  increasingly  strong  mandates  to  the 
Department  of  Defense  for  more  testing  of  new  weapon  systems  before  those  systems  are 
put  into  production  and  delivered  to  the  operational  forces.  Such  testing  takes  time,  and 
the  consequent  delays  in  start  of  high-rate  production  are  not  cost  free.  Delays  increase 
the  direct  cost  of  production,  and  delay  delivery  of  the  new  system  to  the  operating 
forces.  The  hypothesis  of  this  study  is  that  there  must  be  some  point  in  time  where  the 
likely  costs  of  further  delay  exceed  the  likely  benefits  of further  risk  reduction  through 
more  testing.  This  report  describes  a  conceptual  model,  and  an  associated  analysis 
process,  applied  to  the  F-22  program  to  help  answer  the  question  of  how  much  testing  is 
**  enough  ”  to  justify  authorizing  start  of  high  rate  production  of  that  system.  ” 

RAND  attempted  to  develop  an  analytic  approach  for  using  test  results  to  help  establish  the  timing  of  a 
production  go-ahead  decision.  They  examined  the  type  of  problems  found  during  the  flight  testing  and 
placed  problems  into  one  of  three  categories: 

•  Type  1 :  Potential  Program  Killers  -  these  seriously  diminish  mission  capability  and  are  very  difficult 
and/or  expensive  and  time  consuming  to  correct. 

•  Type  2:  Troublesome  but  Manageable  Problems  -  these  seriously  diminish  mission  capability  but  can  be 
corrected;  however,  they  require  time  and/or  funding  beyond  the  original  program  scope. 

•  Type  3:  Routine  Problems  -  these  can  be  corrected  within  the  original  program  budget  and  schedule. 

RAND’s  analysis  focused  on  Type  1  and  2  problems — ^the  problems  that  have  an  impact  on  cost  and 
schedule. 


^  Giles  K.  Smith,  Use  of  Flight  Test  Results  in  Support  of  F-22  Production  Decision,  RAND 
Corporation,  PM-329-AF,  November  1994. 
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RAND  STUDY  ON  F-22  PRODUCTION  DECISION:  RESULTS 

Findings: 

•  When  major  problems  occur,  it  is  usually  within  first  10-20%  of  full  scale  system 
development  test 

•  No  instances  where  major  problems  were  first  revealed  in  later  phases  of  development 
test  program 

Conclusion: 

Unlikely  that  keying  production  decisions  to  completion  oflOT&E  testing  would 
substantially  reduce  technical  risk 


Using  a  database  of  prior  programs,  RAND  estimated  ways  to  link  production  commitments  to  flight  test 
results.  The  RAND  report  described  the  following  analytic  results: 

•  Type  1  problems  are  rare,  and  when  they  occur  it  is  usually  within  the  first  10-20%  of  full-scale  system 
testing. 

•  There  are  no  instances  when  serious  (Type  2)  problems  were  first  revealed  in  the  later  phases  of  the 
development  test  program. 

•  It  is  unlikely  that  keying  production  decisions  to  completion  of  lOT&E  testing  would  substantially 
reduce  technical  risk. 

To  put  this  in  perspective,  we  looked  at  the  current  schedule  of  the  F-22,  and  at  20%  of  full-scale  system 
development  testing.  The  Lockheed  schedule  calls  for  approximately  25  hours  per  month,  with 
approximately  400  hours  of  flight  testing  scheduled  by  December  1997  (10  months  using  1.5  aircraft),  and 
approximately  1,400  hours  by  December  1998  (using  4  aircraft). 

The  1,400  hours  represent  27%  of  scheduled  flight  test  hours.  At  that  point,  only  the  Long-Lead  items  for 
Lot  1  and  2  and  Lot  1  Contract  Award  will  have  been  committed.  (To  reach  20%  of  flight  test  hours,  the 
F-22  would  have  to  fly  1,040  hours.) 

A  large  portion  of  testing  on  the  F-22  is  testing  of  the  avionics.  Flight  testing  of  avionics  begins  in  August 
1998,  and  20%  of  avionics  flight  testing  is  scheduled  for  completion  by  the  first  quarter  of  1999  (Lockheed 
estimate).  At  that  point,  Long-Lead  items  for  Lot  2(12  aircraft)  and  Contract  for  Lot  1  should  have  been 
awarded. 

Given  the  amount  and  duration  of  testing,  it  seems  unlikely  that  keying  production  decisions  to  completion 
of  dedicated  lOT&E  testing  (February  2002)  would  substantially  reduce  technical  risk.  Further,  it  appears 
that  keying  production  decisions  to  Lot  2  Long-Lead — or  conservatively— Lot  2  Contract  Award,  should 
be  sufficient  to  reduce  risk. 
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The  IDA  study  focused  on  DT&E  and  the  RAND  study  assessed  the  likelihood  of  problems  occurring 
within  DT&E  and  lOT&E.  The  DSB  Task  Force  also  had  available  for  its  review  comparison  data  from  an 
unpublished  source  that  suggested  two  other  possible  concurrency  metrics  that  focused  on  the  end  of 
dedicated  lOT&E  as  the  basis  for  production  decisions.  Dedicated  lOT&E  is  currently  scheduled  for 
completion  approximately  8  months  later  than  completion  of  DT&E,  or  February  2002. 

The  unpublished  data  show  the  number  of  months  from  initial  production  to  completion  of  dedicated 
lOT&E.  The  Task  Force  also  indicated  the  Lot  2  milestones  we  believe  critical  to  the  program.  Lot  2  was 
used  as  a  benchmark  for  comparison  because  Lot  2  is  the  transition  point  from  the  4-aircraft-per-year,  to 
the  12-per-year  production  rate.  Also,  using  the  results  from  the  RAND  study,  the  Task  Force  believes  that 
Lot  2  is  a  better  milestone  upon  which  to  base  production  decisions.  The  two  arrows  on  the  F-22  bar  show 
these  two  milestones:  Long-Lead  award  for  Lot  2  production  and  Contract  Award  for  the  Lot  2  buy. 
Using  this  benchmark,  the  chart  shows  that  F-22  development  compares  favorably  to  that  for  historical, 
fighter  aircraft  in  terms  of  when  major  commitment  to  production  aircraft  is  made  prior  to  the  completion 
of  the  lOT&E  test  phase. 

The  chart  also  shows  the  stretchout  of  the  F-22  program — ^there  is  a  much  longer  delay  (i.e.,  a  longer  test 
program)  for  the  F-22  than  for  other  programs.  This  delay  represents,  inter  alia,  risk  reduction  (more 
testing  with  more  time  to  make  improvements),  funding  shortfalls,  and  the  trend  toward  longer 
procurement  cycles  (which  acquisition  streamlining  initiatives  are  meant  to  reverse). 

It  is  important  to  note  that  the  historical  aircraft  development  programs  considered  in  this  sample  used  Full- 
Scale  Development  (FSD)  aircraft  for  lOT&E  testing.  The  F-22  program  will  use  Pre-Production 
Verification  (PPVI  aircraft  to  conduct  lOT&E  testing.  Therefore,  unlike  the  historical  programs,  the  later 
F-22  production  decisions  will  be  based  on  lOT&E  data  accumulated  on  real  production  aircraft,  not  FSD 
aircraft. 
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TTie  second  concurrency  metric  suggested  in  the  unpublished  source  is  the  comparison  of  the  number  of 
aircraft  procured  prior  to  the  completion  of  lOT&E.  As  the  chart  shows,  a  total  of  80  aircraft  (the  4  PPV 
aircraft  rg.quired  for  lOT&E,  plus  Lots  1-4)  are  to  be  contracted  for  before  completion  of  lOT&E.  In 
addition,  the  Long-Lead  procurement  for  another  48  (Lot  5)  aircraft  is  scheduled  to  take  place  in  December 
2001,  two  months  before  the  scheduled  end  of  dedicated  lOT&E.  Again,  in  comparison  with  the  other 
programs,  the  degree  of  concurrency  does  not  appear  unreasonable. 


SUMMARY  FINDINGS  REGARDING  SCHEDULE  CONCURRENCY  METRICS 


•  F-22  Degree  of  Overlap 

“  Consistent  with  other  tactical  aircraft  programs 

•  Risk  of  Proceeding 

-  Key  date  is  Long-Lead  for  Lot  2,  by  that  point  20%  of  flight  test  program  is 
scheduled  for  completion 

-  Cumulative  cost  for  Lot  1  and  2  Long-Lead  items  plus  Lot  1  (with  support  and 
training  equipment)  is  approximately  $L3B 

•  Cost  of  Delay 

-  SPO  estimate  of  $8-1  OB  additional  cost  over  program  for  multi-year  extension  of 
LRIP 

-  Contractors  estimate  of  $2.3B  if  Lot  2  is  reduced  from  12  to  4  aircraft 


In  summary,  we  looked  at  three  aspects  of  DoD’s  definition  of  concurrency. 

•  Degree  of  Overlap  -  The  IDA  metrics  showed  that  the  degree  of  overlap  is  consistent  with  other  tactical 
aircraft  programs. 

•  Risk  of  Proceeding  -  The  technical  risk  of  proceeding  is  evaluated  using  the  RAND  metric.  The  key  date 
is  Long-Lead  for  Lot  2;  by  that  point,  20%  of  the  flight  test  program  is  scheduled  for  completion.  The 
cumulative  cost  for  Lots  1  and  2  Long-Lead  items  plus  Lot  1  aircraft  (with  support  and  training 
equipment)  is  approximately  $1.3B.  Using  RAND’s  figure  of  the  first  10%-20%  of  full-scale  testing 
during  which  Type  1  problems  are  likely  to  occur,  if  at  all,  and  the  amount  and  duration  of  testing,  it 
seems  unlikely  that  keying  production  decisions  to  completion  of  dedicated  lOT&E  would  substantially 
reduce  technical  risk. 

•  Cost  of  Delay  -  The  cost  of  delay  provided  by  the  F-22  SPO  and  contractors  Lockheed  and  Pratt  and 
Whitney  are  shown  above.  The  cost  of  delay  provided  by  the  F-22  SPO  is  $8B-$10B  over  the  life  of  the 
program  due  to  loss-of-leaming,  inflation,  and  further  HMD  stretch-out  of  testing  and  low  rate 
production  for  3  additional  years  of  LRIP.  In  addition,  Lockheed  and  Pratt  and  Whitney  estimate  that 
reducing  Lot  2  from  12  to  4  aircraft  would  add  $2.3B  (assuming  a  4%  inflation  rate)  to  the  total  cost  of 
the  current  program.  (RAND  estimated  this  cost  impact  at  $600M.) 


14 


TASK  FORCE  EMPHASIS 

•  Review  of  various  concurrency  metrics  did  not  portray  complete  picture 

•  Task  Force  conducted  a  brief,  but  aggressive  top-level  critical  technical  and  management  review 

-  Does  program  recognize  key  technical  challenges? 

~  Does  a  plan  exist  to  monitor  and  manage  these  risk  areas? 

-  Is  testing  and  demonstration  early  enough  to  assess  program  production  readiness? 

-  What  is  effect  on  combat  capability  if  some  or  several  goals  are  not  achieved? 

-  Can  decisions  regarding  production  lot  quantity  buy  be  deferred  until  later  in  EMD? 

•  Lot  2  (12A/C)  Long-Lead  and  especially  Contract  Award  considered  critical  milestones 

•  Key  decision  points  identified  for  each  critical-technical  area 

-  Airframe 
~  Engine 

-  Low  Observables 

-  Radar 

-  Passive  Surveillance  and  Electronic  Warfare 

-  Sensor  Control/Sensor  Fusion/Pilot  Interface 

-  Flight  ControlA^ehicle  Management 

-  Software 


To  assess  the  concurrency  of  the  F-22  program  in  isolation  does  not  give  a  complete  picture  of  the 
potential  technical  or  programmatic  risks  in  the  program.  Therefore,  the  Task  Force  conducted  a 
review  of  the  technical  and  management  status  of  the  program  in  several  key  major 
system/subsystem  areas.  This  was  done  in  an  effort  to  relate  concurrency  and  technical  risk. 

The  costs  of  concurrency  risks  within  the  F-22  program  are  directly  related  to  the  following  four 
types  of  changes: 

•  Major  structural/engine  Long-Lead  changes  (most  expensive) 

•  Fixed  hardware  Long-Lead  changes  (expensive) 

•  Changes  to  existing  Line  Replaceable  Units  (LRUs) 

•  Software  changes  only  (may  be  expensive  in  terms  of  labor  but  no  impact  on 
production  costs) 

Lot  2  Long-Lead  and  Contract  Award  were  chosen  as  the  critical  milestones  to  balance  schedule  and  cost 
commitment,  while  ensuring  the  fielding  of  the  system  in  a  reasonable  time. 
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AIRFRAME 

Status: 

•  Weight  excess  of  the  order  of  1 ,000  lbs.  (relative  to  proposal) 

•  Combination  of  airframe  and  engine  excess  weight  and  excess  specific  fuel 
consumption  (SFC)  result  in  decrements  in  specification  subsonic  mission  and  ferry 
ranges  and  sustained  load  factors 

•  Extensive  wind-tunnel  and  YF-22  flight  tests  provide  confidence  on  engine-inlet 
compatibility  and  stability/control  characteristics 

•  Extensive  structural  component  and  materials  testing  programs  for  risk  reduction 
Concerns: 

•  Weight  control  .  . 

•  Extensive  use  of  new  materials  and  fabrication  processes  and  new  applications  of 

materials 

•  Higher  than  previously  experienced  temperature/time  durations  for  composites  in 
supersonic  flight  legs 

•  1st  lifetime  fatigue  test  scheduled  for  completion  3  months  before  Lot  1  contract 
authorization  and  Lot  2  Long-Lead  procurement 

Key  Events: 

•  Static  Test  Complete  May  1998 

•  1st  Lifetime  Fatigue  Test  September  1998 

Lot  2  Long-Lead  Commit  December  1998 

•  2nd  Lifetime  Fatigue  Test  March  1999 

Lot  2  Contract  Award  November  1999 


Current  estimates  put  the  airframe  weight  at  the  order  of  1,000  pounds  above  the  proposed  weight,  and 
estimates  may  not  yet  have  stabilized.  Special  efforts  are  under  way  to  control  weight;  as  one  example  -  a 
“Tiger  Team”  was  convened  by  the  Program  Office,  and  it  completed  its  report  in  December  1994. 

Extensive  structural  and  materials  testing  (and  corresponding  manufacturing  development  work)  has  been 
conducted  to  reduce  risks  in  these  areas. 

Wind-tunnel  and  YF-22  prototype  flight  tests  provide  confidence  on  the  generally  satisfactory  nature  of 
engine-inlet  compatibility  and  stability  and  control  characteristics.  Also  validated  to  some  degree  were 
aerodynamic  drag  and  other  parameters  along  with  estimating  methods. 

The  current  estimate  of  airframe  and  engine  excess  weights,  together  with  the  estimated  excess  engine 
specific  fuel  consumption,  results  in  estimated  decrements  from  specification  values  of  subsonic  mission 
and  ferry  ranges  and  sustained  load  factors. 


There  is  a  continuing  concern  over  weight  control  and  whether  projected  weight  reductions  throug 
redesign  will  be  achieved,  and  whether  new  estimates  of  excess  weights  of  other  parts  and  equipment  will 
stabilize.  The  “Tiger  Team”  on  weight  control  reviewed  this  problem  and  reported  findings  in  December 

1994. 

There  is  extensive  use  of  new  materials  and  processes  as  well  as  new  applications  of  established  materials, 
including  both  composites  and  cocuring  processes  as  well  as  welded  metal  castings  and  large  forgings. 
While  the  materials  and  structural  component  testing  for  risk  reduction  have  been  comprehensive  and  a 
aspects  of  design  and  testing  conform  to  Air  Force  Structural  Integrity  Program  standards,  only  full-scale 
static  and  fatigue  tests  can  provide  a  sufficient  degree  of  confidence  in  the  structural  integrity  of  the 
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production  configuration  of  the  aircraft  to  warrant  moving  toward  high-rate  production.  Completion  of  the 
static  test  and  1st  lifetime  of  the  fatigue  test  in  September  1998  precedes  by  only  3  months  the  initiation  of 
Lot  2  Long-Lead  Procurement. 

In  terms  of  lifetime  durability  the  higher  average  temperature-time  durations  for  supersonic  flight  mission 
segments  are  more  severe  than  in  prior  composite  applications,  although  these  temperatures  are  not  in 
excess  of  allowable  as  determined  in  laboratoiy  materials  tests. 

Key  Events: 

The  key  events  are  listed  on  the  chart  together  with  Lot  2  Long-Lead  (LL)  and  Lot  2  Contract  Award  (CA) 
dates. 

Conclusions: 

The  critical  airframe  areas  in  F-22  development  are  weight  (including  engine  weight)  and  structural 
validation  in  static  and  fatigue  tests.  Aggressive  weight  control  measures  are  being  carried  out  but 
continued  monitoring  is  warranted.  A  critical  point  in  the  structural  integrity  area  is  the  completion  of  the 
first  lifetime  fatigue  test;  the  schedule  and  progress  in  the  structural  test  program  should  also  be  closely 
monitored. 
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ENGINES 

Status: 

•  Excessive  vibratory  stress  levels  found  in  fan  and  high  pressure  turbine  1 

•  SFC  too  high  -  15%  for  subsonic  mission 

I  •  Fixes  for  stress  problems  require  redesign  of  overstressed  components  and  adjacent 

components  (stress  drivers) 

I  •  Fixes  for  excess  SFC  include  several  component  efficiency  improvements — ^major 

contributors  are  turbine  and  exhaust  nozzle 

•  Estimated  excess  SFC  of  improved  engine  is  1 .8% 

•  Estimated  excess  weight  of  improved  engine  is  120  pounds  I 

■Cppggmg: 

•  Large  amount  of  “fixing”  required 

•  Late  in  program  to  be  working  large  performance  deficiencies 

•  Unlikely  to  meet  all  performance/durability  goals  prior  to  first  flight 

•  Program  provides  for  fall  back  using  interim  engines  in  early  flight  test 

K?y  Events: 

•  First  Test  of  Flight  Test  Engine  Core 

July  1995 

•  First  Test  of  Complete  I  SR*  Configuration 

October  1995 

•  Initiate  Durability  Test  of  Flight  Test  Engine 

December  1995 

•  Begin  Flight  Test  Engine  Assembly 

March  1996 

•  First  Flight  Test  Engine  Delivery 

September  1996 

•  IFR**  Milestone 

December  1996 

•  FFR***  Milestone 

December  1997 

•  ISR*  Milestone,  LRIP  Decision 

December  1998 

•  First  PPV  Lot  Engine  Delivery 

March  1999 

*  Initial  Service  Release 

**  Initial  Flight  Release 

***  Full  Flight  Release 

Status: 

The  engine  has  excessive  vibratory  stress  levels  in  the  hollow-blade  fan,  and  also  in  the  high  pressure 
turbine.  Fixes  have  been  found  involving  both  the  critical  component  designs  and  revisions  in  the 
configurations  of  adjacent  components  to  change  the  vibratory  forcing  functions.  There  may  also  be 
durability  issues  in  other  hot  section  parts  if  temperatures  are  raised  to  meet  performance  goals. 

The  engine  is  also  deficient  in  performance  at  a  number  of  significant  operating  points.  One  of  the  most 
critical  in  operational  terms  is  the  subsonic  cruise  mission  for  which  the  SFC  is  15%  too  high.  The 
proposed  fixes  would  result  in  an  estimated  subsonic  SFC  that  is  1.8%  above  the  current  specification. 
These  fixes  involve  design  changes  to  improve  component  efficiencies  and  reduce  flow  losses  at  several 
locations  in  the  engine.  However,  the  largest  projected  improvements  are  in  major  redesigns  of  the 
turbines  and  the  exhaust  nozzle.  These  significant  component  redesigns  will  not  be  incorporated  in  an  all- 
up  test  engine  until  the  summer  of  1995.  The  current  estimate  of  engine  weight  incorporating  the  fixes  is 
120  pounds  over  the  target  weight  originally  specified. 

Concerns: 

The  number  and  extent  of  the  engine  design  changes  needed  to  correct  performance  deficiencies  is 
considerable.  For  fighter  engines  of  this  type,  the  components  tend  to  be  closely  coupled  as  elements  of 
the  propulsion  system.  Design  changes  in  one  area  may  cause  unanticipated  effects  in  other  areas  of  the 
engine. 
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It  is  highly  challenging  to  be  correcting  significant  shortfalls  in  performance  at  this  time  in  the 
development  cycle,  particularly  if  EMD  engines  are  to  be  available  in  time  for  installation  in  the  first  flight 
test  aircraft  in  September  1996.  Normally,  at  this  stage  of  development,  engineering  and  test  efforts  would 
be  devoted  primarily  to  solving  problems  of  reliability  and  durability  in  the  near-final  configuration  of  the 
production  engines. 

Key  Events: 

As  shown  on  chart. 

Conclusions: 

In  view  of  the  compressed  time  available  for  test  validation  and  refinement  of  the  modified  engine  (July 
1995  availability  of  the  core  engine  for  test  and  October  test  availability  of  a  complete  engine  including 
redesigned  nozzle),  it  appears  that  it  would  be  desirable  to  consider  possible  program  alternatives  in  the 
event  that  unanticipated  events  preclude  availability  of  currently  planned  redesigned  engines  in  time  for 
installation  in  the  earliest  flight  test  aircraft.  It  is  understood  that  the  SPO  has  a  checkpoint  for  this  purpose 
scheduled  for  mid-1995.  Alternatives  range  from  acceptance  of  an  EMD  engine  with  greater  than  1.8% 
excess  SFC  based  on  trade-offs  of  operational  effectiveness,  cost,  compatibility  with  force  modernization 
phase,  and  other  factors,  to  adopting  an  interim  engine  for  some  of  the  flight  test  aircraft  and  revising 
schedules.  Schedule  adjustments  affecting  concurrency  may  also  be  made  at  this  decision  point,  if 
considered  necessary.  ’ 


19 


LOW  OBSERVABLES 

Status: 

•  Many  LO  design  and  manufacturing  challenges  exist 

-  Radar  antenna,  radome,  nose  integration 

-  Gaps,  cracks,  seals,  drains,  etc. 

•  Program  has  plans  to  ensure  that  problems  are  identified  early 

-  Full  scale  RCS  pole  models  (aircraft,  inlet,  canopy) 

-  Production  RCS  imaging  facility 

-  Early  LO  flight  test  (EMD  A/C  #4) 

•  Infrared  signature  receiving  proper  emphasis  based  on  F-22  capability  and  likelihood  of  encounter 
Concerns: 

•  Program  must  maintain  emphasis  on  testing  full  scale  articles  built  with  production 
components/processes 

Key  Events: 

•  Full-scale  aircraft  RCS  pole  model  testing  complete  June  1998 

Lot  2  LL  Commit  December  1998 

•  Early  RCS  LO  flight  tests  complete  March  1999 

Lot  2  Contract  Award  November  1999 

Conclusions: 

•  No  unacceptable  concurrency  risk  exists 


Status: 

During  DEM/VAL  the  program  constructed  a  full-scale  radar  cross  section  (RCS)  pole  model  and 
demonstrated  that  the  design  could  yield  very  impressive  radar  cross-section  signatures.  During  EMD  this 
design  must  be  further  refined  with  attention  to  integration  of  critical  components  as  well  as  development 
of  the  manufacturing  processes  that  are  required  to  yield  a  low  signature  vehicle. 

Many  challenges  exist.  Two  of  the  bigger  ones  are  integration  of  the  radar  antenna  and  radome  into  the 
aircraft  and  attention  to  the  details  related  to  gaps/cracks/doors/seals  and  drain  holes.  These  and  other 
challenges  are  acknowledged  by  the  program  office  and  plans  have  been  instituted  to  reduce  risk. 

The  program  will  be  conducting  low  observable  testing  throughout  the  EMD  schedule.  Full-scale  RCS  test 
fixtures,  to  include  a  full-scale  aircraft  model,  will  be  used  for  these  tests.  The  testing  of  full-scale 
hardware,  and  in  many  cases  full-scale  production  hardware  (antennas,  leading  edges,  radome,  canopy, 
etc.),  will  significantly  reduce  the  development  risk.  Several  programs  that  experienced  problems  either 
did  not  or  could  not  test  at  full-scale.  Full-scale  RCS  pole  model  testing  will  take  place  late  1995  through 
mid-1998. 

The  F-22  production  line  at  Marietta,  GA,  will  have  a  radar  cross-section  measurement  system.  This 
system  will  allow  for  each  EMD  and  production  aircraft  to  be  measured  (ground  diagnostic  RCS  imaging). 
This  will  allow  for  an  early  evaluation  of  production  manufacturing  processes  and  their  ability  to  achieve  a 
low  observable  aircraft. 

Low  observable  flight  testing  will  commence  with  EMD  A/C  #4.  These  tests  are  critical  as  they  will 
demonstrate  the  achieved  radar  cross-section  of  a  flight  vehicle  built  using  the  actual  production  line 
process.  The  initial  results  of  these  tests  should  be  available  in  early  1998. 

The  infrared  signature  is  secondary  in  importance  to  the  RCS  of  the  F-22  in  view  of  the  current  threat 
definition.  This  area  is  receiving  proper  emphasis  and  a  similar  plan  has  been  developed  to  refine  the 
design  and  develop  production  processes. 
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Concerns: 

It  is  critical  that  the  program  maintain  and,  where  possible,  increase  emphasis  on  full-scale  test  articles  that 
are  built  with  production  components  and  processes.  Changes  to  the  current  low  observable  program 
schedule  or  a  reduction  in  the  reliance  on  production  components  or  processes  for  testing  will  significantly 
increase  the  risk  of  concurrency. 

Key  Events: 

In  relation  to  the  Lot  2  Long-Lead  and  Contract  Award  decisions,  two  low  observable  related  significant 
events  exists.  Full-scale  RCS  pole  model  testing  will  be  complete  before  the  Lot  2  Long-Lead  decision  in 
December  1998  and  early  low  observable  flight  testing  will  be  complete  before  the  Lot  2  Contract  Award 
date  of  November  1999.  Data  from  these  events  will  allow  the  F-22  program,  the  Air  Force,  and  OSD  to 
evaluate  the  current  status  of  the  low  observable  design  and  manufacturing  process.  Data  from  these 
events  can  be  used  to  support  go-ahead  decisions  or  alternative  decisions  to  delay  or  reduce  the  Lot  2 
quantity. 

Conclusions: 

No  unacceptable  concurrency  risk  exists  in  the  F-22  low  observable  area.  Reducing  the  Lot  2  production 
quantity  at  this  time  will  not  reduce  the  risk.  Significant  test  data  will  be  available  prior  to  Lot  2  Long- 
Lead  and  Contract  Award  to  more  thoroughly  evaluate  program  risks. 
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RADAR 


Status: 

•  Components  developed,  and  component  performance  said  to  be  acceptable 

•  CIP  throughput  and  memory  demands  within  allocation 

•  Recently  had  “very  successful”  critical  design  review  (CDR) 

Concerns: 

•  Clutter  rejection  still  to  be  demonstrated 

-  Antenna/radome  sidelobe  performance  said  to  be  adequate 

-  Vibrational  effects  on  clutter  rejection  to  be  tested  January  95 

•  RCS  of  antenna  “a  little  high”  -  candidate  fix  to  be  tested  February  95 

•  T/R  modules  -  can  contractors  meet  schedules  of  both  F-22  and  GBR? 


Key  Events: 

•  First  complete  radar 

•  Block  I  software  tested  in 
flying  test  bed  (FTB) 

•  First  radar  flight  in  EMD  A/C 

•  Block  III  software  tested  in  FTB 
Conclusion: 

•  Assuming  near-term  tests  successful. 


-April  1996 
-September  1998 

-October  1998 

Lot  2  Long-Lead  Commit  December  1998 
Lot  2  Contract  Award  November  1999 
-July  2000 

r  hardware  development  proceeding  well 


Status: 

The  F-22  radar  is  a  solid  state  electronically  scanned  multi-mode  radar.  Although  quite  different  from  any 
other  current  operational  radar,  it  builds  on  a  legacy  of  work  on  solid  state  transmit/receive  (T/R)  module 
development  and  the  multi-mode  nature  of  radars  like  that  in  the  B-IB.  The  radar  has  also  benefited  from 
the  long  period  since  its  initial  design  for  the  YF-22/YF-23,  and  the  preceding  Ultra  Reliable  Radar  (URR) 
program.  Even  in  the  YF-22  program  hardware,  the  issues  were  largely  those  of  cost,  especially  of  the  T/R 
modules,  and  not  of  feasibility  or  producibility.  The  radar  signal  processing  is  done  in  the  common 
integrated  processor  (CIP).  It  requires  substantial  amounts  of  throughput  and  memory,  but  this  is  said  to  be 
within  CIP  allocations.  The  real  time  control  of  the  multiple  modes  is  being  worked  in  the  mission 
software  design  and  development.  The  reliability  projection  is  so  good  that  even  a  substantial  shortfall 
would  still  result  in  a  very  maintainable  system. 

Concerns: 

The  Task  Force  has  two  performance  concerns.  The  first  performance  concern  arises  from  the  requirement 
to  detect  low  RCS  targets  against  a  clutter  background.  This  requires  very  good  sidelobe  control  of  the 
antenna/radome  combination.  We  were  told,  but  did  not  verify,  that  the  sidelobe  performance  was 
adequate.  Such  target  detection  in  clutter  also  requires  very  stable  reference  oscillators.  It  is  yet  to  be 
demonstrated  that  the  requisite  stability  is  achieved  in  the  vibration  environment  of  the  aircraft.  The 
second  performance  concern  is  that  the  contribution  of  the  radar  antenna  to  the  aircraft  RCS  is  above  spec. 
This  is  a  matter  of  precision  in  antenna  impedance  matching.  The  contractor  has  an  approach  to  improving 
the  performance  that  will  be  tested  soon. 

There  is  also  a  potential  production  concern  that  should  be  investigated  by  joint  meetings  between  the 
relevant  project  offices.  The  F-22  radar  and  the  Ballistic  Missile  Defense  Organization  (BMDO)  Ground 
Based  Radar  (GBR)  both  make  demands  on  one  of  the  contractors,  Texas  Instruments,  for  T/R  module 
production.  Assurance  should  be  obtained  that  both  programs  are  comfortable  that  the  potential  impact  of 
the  other  on  their  program  schedule  is  negligible  or  acceptable. 
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Key  Events: 

The  event  schedule  shows  that  the  radar  hardware  will  be  delivered  and  installed  prior  to  Long-Lead 
commitment  to  Lot  2  and  will  fly  in  an  EMD  aircraft  before  Lot  2  Contract  Award.  Thus,  there  appears  to 
be  little  risk  in  the  radar  hardware  concurrency. 

Conclusions: 

Assuming  near-term  tests  will  be  successful,  radar  hardware  development  is  proceeding  well. 
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PASSIVE  SURVEILLANCE  AND  ELECTRONIC  WARFARE 


Multiple  (15-20)  sources  of  passive  surveillance  designed  to: 

•  Provide  situational  awareness 

•  Minimize  active  (radar)  emissions 

•  Provide  identification 

•  Cue  defensive  measures  -  weapons,  electronic  warfare  (EW),  flares 
Status: 

•  CNI  and  EW  CDR  successfully  completed 

•  Fusion  algorithms  prototyped,  sample  scenarios  simulated,  critical  antennas  prototyped 
Concerns: 

•  Major  advances  in  automated  sensor  management,  multi-source  integration  and  ESM 
system  performance 

Integrated  architecture  helps 

-  Can  back-off  on  performance  of  selected  inputs 

-  Must  achieve  integration 

-  Potential  antenna  aperture  problems  are  a  concurrency  concern 

•  Algorithms  to  deal  with  all  the  signal  complexities,  variabilities,  and  unexpected  or 
conflicting  events  and  parameters  will  be  a  significant  challenge 

Key  Events: 

•  Proof  of  implementation  progress  will  come  during 

-  Block  3  Avionics  Integration  -  August-December  1999 

-  Flying  Test  Bed  &  Aircraft  Testing  -  October  1999-March  2000 

•  This  is  after  Lot  2  Long-Lead  and  coincident  with  Lot  2  Contract  Award 
Conclusions: 

•  Very  challenging  area  with  the  attendant  risks 

•  Proceeding  well  and  being  carefully  managed 

The  F-22  has  many  means  of  passively  receiving  situation  awareness  information  from  both  its  own 
sensors  and  external  sources.  The  intent  is  to  allow  the  pilot  to  see  the  threat  situation  while  minimizing 
use  of  the  active  radar.  These  inputs  will  also  help  in  identifying  the  threats  and  providing  cues  to  the 
weapons,  electronic  warfare  systems,  and  flares. 

The  passive  surveillance  system  includes  many  stressing  performance  requirements.  Most  are  well  beyond 
anything  previously  accomplished  on  an  airborne  platform  regardless  of  size.  Particular  areas  of  advance 
include  automated  sensor  management,  multi-source  integration,  electronic  support  measures  (ESM) 
system  performance,  and  target  handling  capabilities.  These  are  major  steps  forward  with  the  attendant 
risks. 

Failure  to  achieve  the  current  full  specified  performance  in  any  one  passive  subsystem  would  not  be  fatal 
since  so  many  potential  information  sources  are  involved.  In  the  worst  case,  the  active  radar  could  be  used 
more.  However,  that  would  increase  the  aircraft’s  observability  against  sophisticated  threat  systems. 

To  be  successful,  the  integration  of  the  multiple  sources  must  be  achieved.  Presented  individually,  the  data 
would  clearly  overload  the  pilot. 

Status: 

The  current  approach,  progress,  and  plans  appear  sound.  There  are  early  efforts  to  identify,  track,  and 
address  the  key  risks.  Prototyping  of  the  most  stressing  areas  has  begun.  Fusion  algorithms  have  been 
written  and  are  being  tested  under  different  scenario  loads.  Testing  and  simulation  is  planned  in  dense 
emitter  environments.  Green  Flag  testing  would  stress  the  system  considerably. 
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Concerns: 

The  primary  risk  relating  to  production  concurrency  appears  to  be  from  problems  with  the  antenna 
apertures.  If  the  system  experiences  field-of-view/obstruction  problems,  lacks  sensitivity,  or  has  co¬ 
channel  interference,  fixing  the  antennas  could  be  very  expensive  -  given  the  aircraft  would  already  be  in 
production,  and  the  observability  testing  that  would  have  to  be  redone. 

There  will  likely  be  numerous  software  related  problems  found  during  integration  and  testing,  particularly 
related  to  dealing  with  the  variability  of  real  emitters.  These  software  problems,  however,  should  not  have 
an  impact  on  the  production  concurrency  issues  being  addressed  here. 

Key  Events: 

Key  events  in  the  integration  of  the  passive  surveillance  and  EW  systems  are  in  the  Block  3  ground-based 
mtegration  in  August-December  1999  and  then  on  the  Flying  Test  Bed  and  aircraft  early  in  2000.  These 
events  occur  after  Long-Lead  is  released  on  Lot  2  and  coincident  with  Contract  Award  for  Lot  2. 


Conclusions: 

The  requirements  for  this  area  are  a  major  step  forward  in  both  ESM  system  performance  and  automated 
processing.  Good  progress  is  being  made  in  the  design,  and  the  program  is  being  carefully  managed.  This 
area  must  receive  continuing  close  attention  due  to  the  potential  risks  and  rather  late  scheduling  of  the  key 
tests  relative  to  Lot  2  production. 
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SENSOR  CONTROL/SENSOR  FUSION/PILOT  INTERFACE 

Motivation: 

•  Complex  set  of  sensors  in  single  seat  aircraft 

•  Computer  control  necessary  to  manage  workload 
Status: 

•  Control  and  fusion  algorithms  being  developed  and  tested  against  simulated 
threats/scenarios 

•  Cockpit  simulator  built;  being  used  for  interface  evaluation 
Concerns: 

•  Breaking  new  ground  in  pilot/sensor  interface 

•  Software  must  work  in  all  scenarios: 

—  Accommodate  changes  in  enemy  tactics 

-  Provide  easy  response  to  threat  evolution 

•  Will  require  long  evolutionary  software  development 

Pilot  interface  will  also  evolve  -  could  necessitate  hardware  changes  (probably  not 
beyond  LRU  level) 

Significant  Events: 

•  First  flight  of  complete  avionics  in  EMD  aircraft  -October  1 999 

Lot  2  Contract  Award  —November  1 999 

•  First  software  (Block  III)  with  significant  levels  of  sensor  control  and  fusion  tested  in 
FTB  -July  2000 

Conclusion: 

•  An  extended  program  in  developing  and  updating  mission  software  will  be  required; 
pilot  interface  hardware  may  need  to  be  changed.  This  will  probably  not  be  known  by 
Lot  2  Contract  Award. 


The  F-22  incorporates  a  large  number  of  sensors,  many  with  an  assortment  of  operating  modes.  It  has  a 
single  part-time  sensor  operator  (the  pilot).  These  circumstances  dictate  that  1)  the  real  time  management 
and  control  of  the  sensors  must  be  automated,  2)  the  data  on  each  target  from  all  relevant  sensors  must  be 
correlated  and  its  meaning  extracted  by  a  computer,  3)  the  reduced  data  must  be  presented  to  the  pilot  in  a 
user  friendly  way,  and  4)  the  required  reaction  of  the  pilot  be  facilitated  by  good  engineering  of  his 
controls  and  switches.  These  tasks  will  be  done  on  the  F-22  by  computer  to  a  degree  unprecedented  at 
least  in  tactical  aircraft.  ’ 

The  result  should  be  an  impressive  and  very  useful  capability.  Reaching  that  end  will  require  mission 
softw^e  development  process  of  1)  identifying  threat  signatures  and  tactics,  both  nominal  and  plausible 
variations  from  nominal;  2)  crafting  the  procedures  and  algorithms  to  be  used  for  situational  awareness, 
threat  assessment,  and  weapons  control;  and  3)  generating  and  debugging  the  continuing  software 
evolutions.  This  is  well  recognized  throughout  the  F-22  organization.  They  hope  and  believe  that  the 
evolution  can  be  confined  to  the  software. 

We  caution  that  it  seems  likely  that  as  large  numbers  of  pilots  learn  the  F-22,  there  may  arise  pressures  to 
change  displays  and  switch  function/locations.  Doing  so  may  involve  more  than  software  changes,  despite 
good  efforts  to  provide  system  flexibility. 

Conclusion: 

The  issues  of  control/fusion/interfaces’ are  not  thoroughly  tested  until  the  Block  3  software  is  available,  at 
least  to  the  FTB.  The  Block  3  FTB  tests  finish  about  6  months  after  the  Lot  2  Contract  Award.  It  seems 
very  likely  that  further  changes  in  software  will  be  required  and  quite  possible  that  cockpit  hardware 
changes  may  be  desired.  Hopefully,  and  most  likely,  these  could  be  contained  to  the  LRU  level. 
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FLIGHT  CONTROL/VEHICLE  MANAGEMENT 

Concerns: 

•  None  -  Risks  are  low 
Status: 

•  Stability  and  Control  flight  tests  occur  early  with  few  production  commitments 

-  Testing  is  60%  complete  with  8  aircraft 

-  100%  complete  with  20  aircraft 
Conclusions: 

•  Success  can  be  reasonably  expected 

-  Design  based  on  proven  FEW  technology 

-  New  features  demonstrated  on  prototypes 
»  aerodynamics,  supercruise,  RCS 

»  pitch  thrust  vectoring 
»  high-alpha  maneuvering 

-  Important  lessons  learned  from  FY-22  mishap 
»  more  rigorous,  reliable  EMD  design  process 
»  better  integrated  aero/thrust  vectoring 


Status: 

The  basic  design  approach  uses  proven  fly-by-wire  (FEW)  concepts  from  previous  programs.  The  major 
differences  are  a)  somewhat  unusual  aerodynamics  due  to  RCS  and  supersonic  cruise,  b)  integrated  pitch- 
axis  thrust  vectoring,  and  c)  maneuvering  at  high  angle-of-attack. 

The  new  elements  of  the  system  have  been  previously  demonstrated  in  DEMA^AL  on  the  prototype  YF-22 
aircraft.  For  example,  aerodynamic  stability  and  control  characteristics,  which  form  the  essential  database 
behind  the  control  system  design,  are  accurately  predicted  for  the  prototype  by  computational  models  and 
wind-tunnel  tests.  Since  the  F-22’s  external  planform  is  substantially  similar  to  the  prototype,  we  can 
expect  the  same  high  modeling  fidelity  in  EMD.  Thrust  vectoring  and  high-alpha  maneuvering  were  also 
demonstrated  on  the  prototype,  although  vectoring  was  not  as  well  integrated  into  the  rest  of  the  control 
system  as  it  will  be  on  the  production  aircraft. 

The  EMD  flight  control  design  approach  effectively  incorporates  important  lessons  learned  from  a  mishap 
in  the  DEMA^AL  program.  One  of  the  prototype  vehicles  exhibited  severe  pitch  oscillations  on  a  max 
power  go-around,  resulting  in  a  “gear-up  landing”  and  subsequent  fire.  Post-accident  investigations 
confirmed  that  this  incident  was  not  caused  by  hardware  failures.  Instead,  the  prototype’s  flight  control 
algorithms  made  the  closed-loop  aircraft  PlO-prone  (PIO  stands  for  pilot-induced  oscillations— a 
misnomer,  since  the  aircraft  is  really  at  fault,  not  the  pilot).  Transient  changes  in  thrust  vectoring  and  gear 
settings  on  the  specific  go-around  triggered  the  oscillation  event.  It  turns  out  that  several  analytical  control 
metrics  exist  which  predict  PlO-tendencies,  and  better  control  algorithms  can  readily  be  designed.  The 
modified  EMD  design  approach  makes  good  use  of  these  metrics  and  is,  in  general,  a  much  more  rigorous 
and  reliable  process  for  designing  the  flight  control  system.  It  also  includes  an  elegant  integration  of  the 
aerodynamic  control  and  thrust  vectoring  features  of  the  aircraft,  which  is  much  improved  over  the 
prototype. 

Conclusions: 

The  flight  control/vehicle  management  system  appears  to  be  in  good  shape. 


27 


SOFTWARE 


Observations: 

•  Common  Development  Process/Tools 

•  Good  Digital  Architectural  Design 

•  Available  Core  Hardware  and  Software 

•  Four  Major  Block  Releases  of  Software 

•  Extensive  Ground  and  Flight  Test 

•  Early  Block  Releases  Address  Greatest  Production  Risks 

•  AFOTEC  Operational  Assessments  Concurrent  With  Development  Flight  Test 

•  Software  Block  Schedules  Must  Be  Met  To  Achieve  Production  Schedule 


Common  Development  Process/Tools: 

A  common  software  development  process  and  software  engineering  environment  have  been  imposed  for 
use  in  the  F-22  program.  Contractors  have  accepted  this  uniform  approach  and  have  made  provisions  for 
the  corresponding  requisite  training  to  ensure  consistency  in  implementation. 

The  software  development  process,  which  represents  a  tailored  DoD-STD-2167A  approach  to  software 
development,  consists  of  methods  that  were  defined  by  the  contractor  Integrated  Product  Team  (IPT). 
Compliance  to  this  process  is  monitored  by  a  robust  set  of  software  metrics,  as  well  as  software  tools  that 
automate  some  aspects  of  the  process. 

The  use  of  the  common  Software  Engineering  Environment  (SEE)  enhances  team  communication  and 
facilitates  system  integration.  The  environment  is  comprised  of  both  commercial  off-the-shelf  (COTS) 
software  products  as  well  as  team  developed  software  tools.  The  SEE  has  been  baselined  and 
modifications  to  its  various  components  are  subject  to  configuration  control.  In  addition,  the  use  of  a 
common  interface  database  ensures  interface  consistency  throughout  the  development  process. 

Good  Digital  Architectural  Design: 

An  open  systems  approach  has  been  applied  to  the  design  of  the  digital  architecture.  The  hardware 
architecture  has  been  designed  in  a  modular  fashion  that  enables  the  addition  of  processing  components  to 
the  backplane  without  requiring  any  software  modifications. 

Similarly,  the  modular  approach  to  the  design  of  the  software  enables  the  inclusion  of  additional 
functionality  without  adverse  impact  to  the  current  design/capability. 

Available  Core  Hardware  and  Software: 

Core  hardware  and  software  (including  the  common  operating  system  and  SEE  tools)  are  now  available  for 
use,  providing  the  basis  for  development  of  the  application  specific  aspects  of  the  program. 

Four  Major  Block  Releases  of  Software: 

An  evolutionary  approach  to  software  development  is  implemented  via  four  block  releases  (0-3)  each  of 
which  provides  greater  functionality  by  building  on  the  previous  release. 

•  Block  0  provides  the  flight  control  software,  full  IRS  functionality,  all  operating 
systems,  stores  management,  and  basic  CNI  functions. 

•  Block  1  provides  mission  data  edit  and  event  dates,  single  sensor  control  and 
display,  initial  radar  functionality,  additional  CNI  capability,  and  fusion. 

•  Block  2  provides  additional  radar  and  CNI  capability  and  initial  functionality  in  EW 
and  weapon  stores  management. 
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•  Block  3  represents  the  extension  and  integration  of  functions  provided  in  the 
previous  blocks,  in  addition  to  an  embedded  training  capability. 

Extensive  Ground  and  Flight  Test: 

The  F-22  program  provides  ample  opportunity  for  detecting  potential  software  problems  through  an 
extensive  incremental  process  that  includes  both  ground  and  flight  test.  The  stages  of  this  process  are: 

•  Software  Testing.  This  stage  includes  all  aspects  of  testing  during  the  actual 
software  development  process  (i.e.,  code  and  unit  test,  integration  test,  quality 
assurance  test,  IV&V,  etc.). 

•  Sensor  Stand  Alone  Test.  This  stage  enables  testing  of  the  software  with  actual 
sensor  input. 

•  Avionics  Integration  Lab.  This  stage  facilitates  integration  testing  of  the  software 
within  the  full  avionics  system,  supported  by  simulation  as  required. 

•  Flying  Test  Bed.  This  stage  enables  testing  of  the  software  in  a  “near”  real-time 
environment,  facilitated  by  exercising  and  fusing  actual  sensor  data. 

•  Flight  Test  Aircraft.  This  stage  provides  an  operational  test  environment  in  which  to 
fully  exercise  within  as  realistic  an  environment  as  possible  the  full  software 
functionality,  including  error  detection,  and  system  bounds. 

Early  Block  Releases  Address  Greatest  Production  Risks- 

The  costs  of  concurrency  risks  within  the  F-22  program  are  directly  related  to  the  following  four 
types  of  changes: 

•  Major  structural/engine  Long-Lead  changes  (most  expensive) 

•  Fixed  hardware/ database  Long-Lead  changes  (expensive) 

•  Changes  to  existing  LRUs 

-  New  LRUs  must  be  developed,  or  existing  LRUs  modified 

-  More  existing  LRUs  must  be  built  for  added  processing  power  (less  expensive 
impacts  on  production) 

•  Software  changes  only  (may  be  expensive  in  terms  of  labor  but  no  impact  on 
production  costs) 

With  regard  to  F-22  software,  testing  of  Block  0  will  address  the  major  structural  issues  (most  expensive 
types  of  changes.)  Blocks  1-3  will  address  the  remaining  categories  of  potential  changes.  Block  3.1, 
which  will  address  software-specific  issues  only,  is  unlikely  to  surface  any  changes  other  than  those  that 
impact  the  software  directly.  Accordingly,  the  greatest  concurrency  risks  should  be  identified  by  the  early 
block  releases.  The  last  block  release  should  have  little,  if  any,  impact  on  production  aircraft. 

AFOTEC  Operational  Assessments  Concurrent  With  nevelopment  Flight  Test: 

During  the  software  development  and  testing  process,  the  Air  Force  Operational  Test  and  Evaluation 
Center  (AFOTEC)  will  be  conducting  operational  assessments.  The  results  of  these  assessments  should 
identify  potential  lOT&E  problems  long  before  the  official  lOT&E  occurs.  In  fact,  AFOTEC  input  will  be 
taken  into  consideration  prior  to  the  LRIP  decision  currently  scheduled  for  December  1998. 

Software  Block  Schedules  Must  Be  Met  to  Achieve  Production  Schedule: 

A  basic  assumption  used  in  the  formulation  of  the  software  observations  is  that  the  schedules  currently 
planned  for  software  development  must  be  met,  and  that  the  functionality  to  be  provided  within  each  of  the 
Blocks  as  currently  defined  must  be  retained.  If  a  slip  occurs  within  the  software  development  schedule, 
the  resultant  slip  may  adversely  impact  production. 
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SOFTWARE  (Continued) 

Status: 

•  Requirements  reasonably  solid 

•  CDR  currently  underway 

•  Contractors  moving  into  code/unit  test  stage 
Concerns: 

•  Maintaining  the  software  schedule 
Key  Events: 

•  Release  of  Block  3  software  to  flight  test  (29  September  1999)  should  precede  the 
Contract  Award  for  Lot  2  production 


Status: 

The  mission  software  requirements  including  interface  requirements  are  reasonably  solid  having  used  the 
automated  requirements  and  top  level  design  methodology  (ADARTS).  All  contractors  are  using  these 
common  tools.  Detailed  design  is  well  underway  and  at  the  time  of  this  writing  (7  December  1994)  the 
vehicle  Critical  Design  Review  is  in  progress.  The  contractors  have  begun  to  move  into  the  code  and  unit 
test  phase  of  the  mission  software. 

Concerns: 

The  major  software  concern  is  that  of  maintaining  the  block  release  schedule  with  the  functionality 
planned.  With  over  30  contractors  contributing  to  the  1.4  million  source  lines  of  code,  it  is  important  to 
have  enough  of  the  proper  resources  applied  to  ensure  that  the  delivery  schedule  can  be  met  without 
compromising  the  discipline  of  a  good  development  process. 

Key  Events: 

A  key  milestone  event  is  the  release  of  Block  3  software  to  flight  test  (29  September  1999).  It  is  important 
that  this  event  precede  the  Lot  2  production  Contract  Award  scheduled  for  late  1999.  A  review  (discussed 
later)  must  validate  the  observation  that  Block  3  flight  test  will  have  minimal  impact  on  the  Lot  2 
production  configuration. 
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SOFTWARE  (Concluded) 

Conclusions: 

•  The  software  block  releases  are  such  that  the  earlier  releases  test  those  functions  with 
the  highest  cost  risk  to  production  changes 

•  A  review  of  Block  3  functionality  reveals  a  low  likelihood  of  any  significant  cost 
impact  on  production  aircraft  that  would  not  have  been  identified  by  earlier  ground  or 
flight  test  or  by  operational  assessments  by  AFOTEC 

•  The  software  development  among  all  the  contractors  must  be  carefully  planned  and 
replanned  to  keep  critical  paths  through  the  development  on  schedule 

•  A  review  of  the  following  is  prudent  prior  to  Lot  2  Contract  Award  for  Production  to 
assess  the  concurrency  risk: 

-  Block  1  &  2  flight  test  results 

-  Block  3  Avionics  Integrated  Laboratory  results 

-  AFOTEC  operational  assessments/plans 


Block  0  software  release  in  mid- 1996  essentially  permits  the  flight  test  program  to  validate  major 
structural,  aerodynamic,  and  safety  objections  early  in  the  flight  test  program.  The  results  along  with  static 
test  and  fatigue  test  results  should  feed  into  the  LRIP  decision  in  late  1998  that  influences  the  Lot  2  aircraft 
Long-Lead  procurement  decision. 


The  flight  test  of  Block  1  and  2  software  functionality  should  exercise  the  avionics  hardware  to 
demonstrate  most  of  the  individual  functions  the  hardware  is  to  perform.  Any  major  hardware  changes 
should  become  evident  during  this  phase. 

Block  3  functionality  essentially  integrates  the  functionality  that  has  been  previously  tested.  Avionics 
Integrated  Laboratory  (AIL)  testing  and  earlier  testing  should  have  identified  any  major  hardware  changes. 
The  flight  test  of  Block  3  would  most  likely  point  to  software  changes,  or  the  need  for  more  processing 
capability.  Changes  of  this  sort  would  have  only  very  minor  cost  impacts  on  the  production  aircraft.  It  is 
important  to  know  that  the  testing  has  been  in  the  right  “ball  park”  concerning  OT&E.  One  can  rely  on 
concurrent  “operational  assessments”  being  conducted  by  AFOTEC  to  make  this  latter  assessment. 

An  activity  net  of  all  the  key  software  development  tasks  among  all  the  contractors  with  their  detailed 
planning  would  be  useful  in  determining  the  critical  path  through  all  the  contractor  activities  that  determine 
overall  schedule.  Since  the  process  is  dynamic,  the  critical  paths  must  be  reassessed  frequently  with  the 
objective  of  identifying  key  problems,  finding  resources  across  the  IPT  to  resolve  those  problems,  and 
containing  critical  path  extensions  without  compromising  a  good  engineering  process.  A  central  design 
and  implementation  focus  on  the  software  progress  would  be  very  useful. 

A  key  review  should  be  held  prior  to  the  Lot  2  Production  Contract  Award  after  reviewing  Block  1  and  2 
flight  test  results.  Block  3  ground  test  results,  and  the  AFOTEC  operational  assessments  and  plans.  This 
review  can  verify  the  observation  that  there  are  only  minor  production  cost  impacts  in  proceeding  with 
Lot  2,  if  that  is  the  case.  If  not,  then  a  more  optimal  Lot  2  buy  could  be  made.  The  review  point  is  shown 
during  late  1999  by  the  asterisk  on  the  top  line  of  the  present  schedule  chart  on  Page  3  and  the  key  events 
chart  on  the  next  page. 
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F-22  ACQUISITION  KEY  EVENTS 


Calendar  Year 


•  FIRST  FLIGHT 


1995  1996  1997  1998  1999 


V  . 


2000  2001 


•  DEVELOPMENT  KEY  EVENTS  Initial  Test - 

-  Structures  Redesigned 

Turbine 

»  Engines 

-  Low  Observables 


-  Passive  Surveillance/EW 

-  Sensor  Control/ 
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-  Flight  Controls 
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-  Lot  3  (24  A/C) 


IFR  >  Initial  Flight  Release-Engine;  ISR  -  Initial  Service  Release-Engine;  LRIP  -  Low  Rate  Initial  Production:  AIL  -  Avionics  Integration  Laboratory  -  Software 


The  F-22  Acquisition  Key  Events  as  determined  by  the  Task  Force  are  presented  on  this  chart.  First  flight, 
development  key  events,  and  production  key  events  are  displayed.  The  critical  milestones  for  Lot  2  Long- 
Lead  and  Lot  2  Contract  Award  span  the  year  1999,  which  is  shown  shaded  in  the  chart.  For  each 
technical  area,  there  are  several  key  test  events  prior  to  Lot  2  Long-Lead  and  Contract  Award  for 
production  that  can  be  used  to  measure  progress. 
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GENERAL  OBSERVATIONS: 

•  Short  time/limited  effort  by  Task  Force:  6  days  total,  2  at  contractors 

•  Overall  program  appears  well  structured  and  managed 

•  Top  government  and  contractor  people  impressive 

•  The  “spec”  system  provides  major  advances  in  capability  across  the  board 

-  Hence,  less  than  spec  performance  in  certain  areas  will  still  result  in  very  capable  system 
~  As  program  progresses,  this  permits  performance  to  be  traded  for  cost/schedule 


The  Task  Force  had  a  limited  amount  of  time  to  carry  out  its  work.  A  total  of  six  (6)  days  of  meetings 
were  held;  two  in  Washington,  D.C.,  receiving  briefings  on  the  program  and  relevant  studies;  two  at  the 
contractor’s  plant  reviewing  the  major  subsystems  in  detail;  and  two  in  Washington,  D.C.,  reviewing 
findings  and  drawing  conclusions.  There  was  also  much  additional  data  gathering  and  consultation  by 
individuals. 

In  spite  of  this  limited  effort,  the  Task  Force  believes  it  gained  sufficient  understanding  of  the  program  to 
be  confident  its  findings  are  sound. 

The  overall  program  appears  well  structured,  sound,  and  well  managed.  We  were  favorably  impressed  by 
the  knowledge  and  objectivity  of  both  the  government  and  contractor  staff  with  whom  we  interacted. 

The  “spec”  F-22  system  provides  significant  advances  in  every  important  aspect  of  a  tactical  aircraft. 
“Less-than  spec”  performance,  if  encountered  in  certain  areas,  will  still  result  in  a  system  that  will  be  a 
large  step  ahead  of  current  U.S.  and  likely  opponent  capabilities  for  a  long  time,  and  will  thus  continue  the 
“overwhelming  superiority”  of  U.S.  forces  in  this  area. 

As  the  program  progresses,  any  shortfalls  in  sub-area  performance  can  be  reviewed  and  appropriate  trade¬ 
offs  made  with  cost  and  schedule. 
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PLANS/OPTIONS  TO  MITIGATE  RISKS 

•  No  completely  independent  alternative  approaches  for  big  ticket  items 
(e.g.,  engine/radar)  exist  nor  are  any  recommended 

•  Alternate  approaches  in  parts/sub  areas  are  being  pursued  in  some  cases  and  more  are 
doable  within  the  program  plan 

•  The  general  approach  should  be  to  handle  problems  by  accepting  some  reduced 
performance,  slip  in  schedule,  or  both,  depending  on  risk/cost  trade-offs 


The  Task  Force  was  tasked  to  address  the  issue  of  viable  plans/options  available  to  mitigate  high  risk  areas. 

We  found  that  there  were  no  alternative  completely  independent  approaches  being  pursued  for  the  major 
subsystems  (such  as  the  engine  or  the  radar)  and  we  concluded  that  such  alternate  approaches  were  neither 
practical  nor  needed. 

There  are,  of  course,  alternate  approaches  being  pursued  or  reviewed  at  the  part/component  level  (e.g., 
turbine  blades),  and  others  (e.g.,  LO,  CNl  antennas)  can  be  accommodated  within  the  program’s  plan  and 
schedule. 

We  concluded  that  the  best  approach  to  handling  problems  that  do  arise  is  to  seriously  consider  accepting 
some  reduced  F-22  performance,  slip  in  the  schedule,  or  some  combination  of  both.  As  has  been  noted 
elsewhere  in  this  report,  F-22  performance  in  many  areas,  if  it  fell  short  of  that  specified,  would  still 
represent  a  major  increase  in  military  capability.  Rigid  adherence  to  the  current  specifications  and  goals 
would  generally  be  unproductive  and  very  costly. 
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SUMMARY  CONCLUSIONS 

•  The  program  is  very  ambitious  technically 

•  For  each  risk  area  there  are  significant  achievements  that  should  be  demonstrated  before 
release  of  funding  for  Lot  2  (12  aircraft)  Contract  Award 

•  The  engine  and  the  passive  surveillance  avionics  are  the  highest  risk  areas 

•  In  the  event  of  inadequate  progress,  the  program  can  be  slipped  by  staying  at  the  4  aircraft 
per  year  rate 

•  Stretching  the  program  in  this  way  may  reduce  risk  but  can  create  cost,  manpower,  and 
obsolescence  problems 

•  There  is  no  risk-concurrency  reason  to  introduce  such  a  stretch  at  this  time 


With  its  many  “first  time”  features  for  a  fighter  (super  cruise,  vectored  thrust,  stealth,  passive  location 
system,  integrated  avionics),  the  F-22  is  one  of  the  most  challenging  developments  undertaken  in  recent 
times.  The  overall  program,  starting  with  the  prototype  in  the  DemonstrationA^alidation  Phase  and  related 
R&D  efforts,  has  been  structured  to  address  all  these  areas  in  an  orderly,  appropriate  way  and  much  has 
been  accomplished. 

From  the  standpoint  of  risk/concurrency,  the  critical  point  in  the  program  is  the  ramp-up  in  the  production 
program  from  4  aircraft  to  12  aircraft  per  year  (Lot  2). 

The  Task  Force  identified  for  each  risk  area  specific  significant  events  whose  accomplishment  should  take 
place  before  release  of  Contract  Award  (and  to  a  lesser  degree  Long-Lead)  funding  for  Lot  2,  the  12 
aircraft  buy. 

It  should  be  noted  that,  at  this  time,  the  engine  and  the  Passive  Surveillance  avionics  subsystem  have  the 
greatest  number  of  uncertainties.  Consequently,  if  there  are  changes  in  the  program  plan,  emphasis  on 
these  two  areas  should  receive  special  attention  and  continued  support. 

If  there  are  significant  delays  in  accomplishment  of  these  key  events  or  if  the  performance  levels  achieved 
are  unacceptable,  the  program  could  be  adjusted  by  staying  at  the  4  aircraft  per  year  production  rate  for  an 
additional  year. 

It  is  noted  that,  while  stretching  a  program  may  reduce  risk,  if  stretching  is  not  necessary  it  can  result  in 
increased  total  costs,  possible  loss  of  key  manpower  and  suppliers,  and  earlier  technical/operational 
obsolescence. 

After  careful  review  of  the  program  plans  and  accomplishments,  the  Task  Force  concluded  that  there  are 
appropriate  future  milestones  on  which  to  Judge  readiness  for  production  ramp-up.  Thus,  there  is  no  reason 
based  upon  risk/concurrency  to  introduce  a  schedule  stretch  at  this  time. 


35 


APPENDIX  A 


TASK  FORCE  MEMBERS 


MEMBERSHIP 


Chairman  DSB  Liaison 

Charles  A.  Fowler  L/Col  Keith  Larson 


Members 
John  Allen 
Alexander  Flax 
Joan  Habermann 
Robert  Nesbit 
Philip  Soucy 
Richard  Sylvester 

Special  Advi$or 
Gunter  Stein 


Co-Executive  Secretary 
Dean  Gissendanner 
Col  George  DeFilippi 

Government  Advisors 
Don  Dix,  DDR&E 
Col  Chuck  Brammeier,  OT&E 
Virginia  Castor,  DDR&E 
Roy  Hempley,  PA&E 
Frank  Traceski,  OASD(ES) 
Jim  Bachand,  OASD(ES) 


Support  Working  Group — Institute  for  Defense  Analyses 

J.  Richard  Nelson 

John  N.  Donis 

Bruce  R.  Harmon 

Joseph  W.  Stahl 

Melanie  G.  Mutton 


APPENDIX  B 


TERMS  OF  REFERENCE 


THE  UNDER  SECRETARY  OF  DEFENSE 


ACQUISITION  AND 
TECHNOLOGY 


3010  DEFENSE  PENTAGON 
WASHINGTON.  DC  20301-3010 


November  1 , 


1994 


MEMORANDUM  FOR  CHAIRMAN,  DEFENSE  SCIENCE  BOARD 

SUBJECT:  Terms  of  Reference  —  Defense  Science  Board  Task  Force 

on  Concurrency  and  Risk  of  the  F-22  Program 


You  are  requested  to  establish  a  Defense  Science  Board  Task 
Force  to  assess  the  degree  of  concurrency  and  risk  in  the  F-22 
program  in  accordance  with  SASC  Report  163-282,  dated  June  14, 
1994.  The  scope  of  your  assessment  should  be  based  on  the  most 
recent  rephasing  of  the  F-22  program.  Work  should  begin  as  soon 
as  possible  with  the  Task  Force  providing  an  interim  report  by 
15  December  1994  and  a  final  report  by  31  March  1995. 

The  Task  Force  should  address  the  following  questions: 

Are  there  any  areas  in  the  F-22  program  of  excessive 
concurrency?  What  is  the  risk  in  each  area? 

For  any  areas  of  identifiable  high  risk,  are  viable 
plans/options  available  that  would  mitigate  the  risk? 

What  conclusions  regarding  F-22  concurrency  and  risk  can  be 
drawn  by  comparisons  to  existing  data  (e.g.,  an  ongoing  IDA 
study)  on  previous  fighter/combat  aircraft  programs? 

The  Director,  Tactical  Warfare  Programs  will  sponsor  the 
Task  Force,  providing  funding  and  other  support  as  necessary. 

Mr.  Charles  Fowler,  will  serve  as  Chairman  of  the  Task  Force. 

Mr.  Dean  Gissendanner  ani  Colonel  George  DeFilippi  of  the  Office 
of  the  Director,  Tactical  Warfare  Programs,  will  serve  as  the 
Task  Force  Co-Executive  Secretaries.  Lt  Col  Keith  M.  Larson, 
USAF ,  will  be  the  Defense  Science  Board  Secretariat 
Representative . 

It  is  not  anticipated  that  the  work  assigned  to  this  Task 
Force  will  cause  any  member  to  be  placed  in  the  position  of 
acting  as  a  procurement  official. 


Paul 


Kaminski 
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TASK  FORCE  SCHEDULE 


TASK  FORCE  SCHEDULE 


3  November  1994  -  Washington,  D.C. 

•  Program  Office  Program  Overview 

•  IDA  Study  on  Concurrency 

4  November  1994  -  Washington,  D.C. 

•  Program  Office  Detailed  Briefings 
-  Airframes,  Engines,  Avionics 

16  November  1994  -  Marietta,  GA 

•  Lockheed  Overview  Briefings 

•  Hangar  Tour  &  Demos 

17  November  1994  -  Marietta,  GA 

•  Lockheed  Detailed  Briefings 

•  Individual  Sessions 

6  December  1994  -  Washington,  D.C. 

•  Task  Force  Discussions 

7  December  1994  -  Washington,  D.C, 

•  Task  Force  Discussions 

•  Brief  Findings  to  Director  Strategic  and  Tactical  Systems 
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